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ABSTRACT 

Department^ hea*ds from 134 institutions :(mainly 
universities) indica^ted the weight they generally give to variouis 
criteria for evaluating ijidivi^ual faculty members. The questionnaire 
they responded; to include'd:^ criteria used for evaluating overall 
faculty performance; sources of information for evaluating teaching; 
and kinds of infbfmation viied for judging scholarship or research 
performance. Although teaching, research, and service are generally 
acknowledged as the major functions of most universities, responses 
:bT the ti53 department heads indicated that piablic or university 
service is usually given little importance in evaluating faculty 
members for decisions regarding tenure, s^alary, and promation. The 
L-r^ea^rcli-univ^ri5it±es— €rmph'^^ research and scholarship. The 
so-cailed doctoral -granting universities and the comprehensive 
universities^ and colleges said that teaching was: ranked first in 
importance^ followed closely by research. The results of the -study 
also suggest an increase in the use of some of the More systematic 
and tangible kinds of evidence for evaluating, teaching performance 
(e.g., student ratings) , though the use of such evidence falls short 
of what department heads think is. needed. In- general, the evaluation 
6f research ail d scholarship depends very much on the level of the 
institution and the type of department. For example, peer judgments 
of research and the number of articles in. ^guality journals are much 
more important in social science departments in the pesearch 
uniWrsities than in. the same departments in the comprehensive 
universities and colleges. (Author) 
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Abstract \ 

\ 

Department heads from 134 institutions (mainly universities) indi- 
cated the weight they generally give to various criteria for evaluating indi- 
vidual faculty members. The questionnaire they responded to included: (a) 
the criteria used for evaluating overalhfaculty performance; (b) the sources 
of-information for evaUiating- teaching; and (c) the kinds of information 
used for judging scholarship; or research performance. Although teaching, 
research, and service are generally acknowledged as the major functions of 
most universities, responses by the 453 department heads indicated that 
public pr university service is usually given little importance in evaluating 
faculty members for decisions regarding. tenure, salary, and promotion. 
The Research Universities emphasize research and scholarship; ns might 
be expected. The so-called Doctocal-Grnnting Universities and the Com- 
prehensive Universities and Colleges said that teaching \vas ranked first in 
importance, followed closely by research. The results of the study also 
suggest an increase in the use of some of themore systematic and tangible 
kinds of evidence for evaluating leaching performance (e,g., student rat- 
ings), though the use of such , evidence falls far short of what departmerit 
heads think is needed. In general, the evaluation of research and scholar- 
ship depends very much on the level of the institution and the type of 
department. For example, peer judgments of research and the numb er of 
ji rticles in q uiility,jQurnals^Tre-muclt-more important in social science de- 
partments in the Research Universities than in the same departments in 
the Comprehensive Universities. and Colleges. - 
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DecibiuPb LUiKerning v tunuru tind prumution arc buLunung 
increasingl) more difhcult to make in Lullugub and uni\ur- 
silies. During the expansion ycar^ of the bixticb, mo.st de- 
partments were concerned largel) uith recruiting and keep- 
ing competent faculty n^embers, but enrollments ha\e ta-^ 
pered off and department*) are no longer^iddiiig staff mem- 
bers. This steady-state eopclition has put many ij^slitutions 
at or near prescribed limits m the pereentage of fa<-ult\ on 
tenure, in fact, the American Assoeiation of Unuersil) Pru- 
tessors* Atmmil Report on th^ UonomiL StatUi, of the Profii^^ion 
found that foO percent of all facultv members had tenure in 
1976.V 

In contrast to the last decade, institutions are now often 
forced to make fine distinctions between generalK compe- 
tent staff members. What intormation do departments use 
in making these decisions? Wow do the critical decision 
makers, such as department heads, think an indi\,uiual fac- 
ulty member's performance ought to be evaluated? These 
questions were investigated in this ;>tui{) b\ sur\e>ing 
pracl ,es at a targe number of universities. 



Previous Research 



A- 



Most studies ot hou lacult) perturmauce is evaluated ha\e 
been based on reports Irom academic c(eags or \ice presi- 
dents. One of the earhest st^u^lie^j^s:by Qustad (1961) and 
included a national samplUj^^f , callege.4 a^d ,uni^ ersiti^^ir^ 
Aslin^and Lee (19b7) repcMjplJ t]|Tc\suriry^^^^ five yejus 

later with a sim^tacly- exti?nsi\e sanj^I^i'oi all tj pes of post- 
secondary institution^." Both stuvliesAisked administrators 
to indicate the importSace of \ariouSy^ourcc\s of j^if^^rmation 
in evaluating, first, facully perfiirmtince anid, s^^<)nft teach- 
ing performance.'^The result.^*^vere analj;z<d b^^t>'pe of in- 
stitution. In both surveys ifiQ universityWnn5.;()f arts and 
sciences ranked. department heads\i}.^;thO;jTj.^,^t important 
source of information. Although thede'{|ns;f*mkiid informal 
student opinion and' systematic stuc^e;T^^«1tings high in 
1961, five years lator they indicated that students were less 
a factor in promotions than they h,;d been. Over 90 percent 
of the arts and science deans ip^Jlhe lJ:Q.,aniyer6ities in the 
Astin and Leesu.'vey indic<]^(pd lifat classroom teaching and 
research were given equaUy^.hlgh vC'eight in assessing fac- 
ulty performance, ."i'"* 

Seldin (1975) repeated the Gustad and the Astin and Lee 
surveys in 1974 with academic de<[)ns in liberal arts colleges 
in.order to examine changesthat might have taken placciin 
the eight-year period- The deans reported more emphasis 



on s>stematic student rajtings^g evaluating teaching (an 
increase from li to 29 percent of ^he colleges) a net slightly 
less emphasis on;research^in evaluating overall facility per- 
formance. The increased use of student ratings iri making 
decisions on facult) advancement was also reported in 
studies that included doctoral-level universities (Bejar, 
1975, Boyd & Schielinger, 1976). A Southern Regional Edu- 
cation Board stud) found, in addition, that the major pur- 
pose of evaluation in doctoral-level universities was to 
make decisions on faculty advancement, rather than to im- 
pro\e instruction, and that department heads bad greater 
responsibility than academic deans or yice- presidents in 
making these decisions (Boyd & Schietinger, 1976). 

One of the feu studies that sampled faculty perceptions 
of evaluation procedures found that the most influential 
factors in promotions and tenure decisions were {^ybiica- 
tions, the depar.(ment head's evaluation, and ^^u'dent rat- 
ings (Thorne, Sebtt, & Beaird, 1976). This study, which was 
conducted at institutions in the Oregon State system, also 
reported ifyii grant support was considered to be influential 
in the pf^^sjcal science departments. . 

AlthoilghMhe past studies point to tKyJimpjKtance of de- 
partment heads in making decisions on advancement, 
none surveyed these individuals' ^perceptions of current 
practice/. Wit'h the exception of the" Oregon State study, the 
earlier std'dies_had also investigated administrators' percep- 
tions of institutionwide procedures rather than the criteria 
^ used at the departmea' level: Faculty performance might be 
viewed quite differently in different fields of study. In par- 
ticular, the evaluatidh pf Research and scholarship, which 
the prior studios had^QOt looked at in detail, mi^ht'be ex- 
pected to vary x:onsicj^HiblyvJ?y department 

Puroose of the Present^Study 

The present stud) report^^ the jmpprtanCe ihat department^ 
heads or chairmen give to varion^' criteria Jn evaluating jr(- 
cfividual faculty memberb,.^siojfdcl>3*P" overall faculty 
performance, the study discussesjhe.ghairmen's.^iews of 
the use and importance of specific c^it^a /or assessing both 
teaching and research perfornriancc\Ttf determine;the ex- 

IBolh tales— Joparlmonl he.idand thajrman—^ippl) to the groups surveyed 
And are usc<l in this report although it is recogni/ed that the ternrs arc not 
ahvays $ynopvnuHi«5 Use otcltaipuan f*ilhor thantiidir;)frj<)n orchatr follows 
current A I' A bly^c guidelines, as published in Ameruan i'>j/i/iu%J5f. June 
1977, p. 492- j 

0 A 



lent to which.currem practices coincide with preferred prac- 
tices,.respondents also were asked to indicate what impor- 
tance they thought stowW.be attached to each criterion. 
^CDmpansons-are- xiiscussed-for-f our departmental sub- 
groups and for three different types of universilies. - 

Procedure 

The questionnaire used in the study (see Appendix A) par- 
tially overlapped the forms first -used in the Gustad (1961) 
and Astin and Lee- (1967) studies and later adapted by Sel- 
diri (1975). It differed.from the forms used jn the previous 
studies in that (1) a question regarding the criteria used to 
evaluate scholarship or research performance was added, 
and (2) the cnteria used to evaluate overalj faculty or teach- 
ing performance were somewhat altered or extended. Four 
copies of the questionnaire were sent to the graduate deans 
of 168 universities (a 50 percent random sample of the 
members, of the Council of Graduate Schools in the United 
States). The deans were asked to distribute the question- 
naires to four heads of departments from a variety of disci- 
plines, giving preference to< departments that offered both 
graduate and undergraduate degree programs. One or 
more usable questionnaires were received from 134 
universities— -a participation rale of about percent. Ques- 
tionnaire returns from deparhnent heads, based- on four 
copies per university, amounted to about 67 percent from 
all universities that were contacted, or 85 percent of the 134 
, universities known to have distributed the questionnaires. 
Bight graduate deans said they could not participate in the 
sliidy. A total of 453 usable questionnaires were included in 

, the data analyses. - 

For purposes of analysis, the universities were grouped 
, according to a classification-scheme developed by the Car- 
negie Commission on tligher l-ducittion (1973), Three 
groups were formed by combining Carnegie subcategories. 
The first level. Research Universities, included (a) the 100 
universities that led in federal financial support in 1969-70 
and-lhal awarded at least 50 doctor's degrees in 1969-70 or 
(b) the 50 institutions that led in terms of total number of 
doctor's degrees awarded during the years I960 to^l970. In 
aii-cases, Ph.D.'s, l-d'.D.'s, M.D/s on the same campus, 
and other doctor's degrees were counted- There were 158 
departments from these Research Universities in the study. 

The second level, Doctoral-Granting Universities, com- 
bined two categories. The first category consisted of institu- 
tions that-awarded 40 or more -doctoral degrees in 1969-70 
or received at least $3 million inrtotal federal financial sup- 
port in eithet 1969-70 or 1970-71. The second category in- 
cluded institutions that awarded at least 10 doctoral degrees 
in 1969-70 and a few new doctoral-granting institutions 
that may be expected to increase the number of degrees 
awarded within a few years. There were 122 departments 
from these Doctoral-Granung Uni\ursitiei> in the stud>, 80 
percent of which were in the first category. 

The third level. Comprehensive Universities and Col- 
leges, consisted mainly of institutions th*it offered a liberal 
arts program and at least two protessiunal or uLcupatiunal 
programs. Many of them ottered a master^-A^legree, but 



lacked a doctoral program or had an e>;treinely limited doc- 
toral program. About 80 percent of the 173 departments in 
this group were at institutions that fit this description. The 
-remaining 2fl percent were at institutions that.offered-a-lib- 
eral arts program and at least one professional/occupational 
program or were specialized institutions (for e>cample, 
schools of .engineering or technology). 

Of the 453 departments in the study, aboiit two-thirds 
were at. public institutions. Sixty-four percent\)f these in- 
stitutions offered a Ph.D.. or other doctoral degree, and 36 
perLcnt offered a master's as their highest degree. Only 3 of 
Ihe 453 departments offered nothing higher than'a 
bachelor's degree. 

The questionnaire respondents represented about 80 dif- 
ferent academic disciplines. To evaluate possible differ- 
ences in the responses of department heads from different 
disciplines, the departments were grouped into four 
categories: Iriimahities, social sciences, natural sciences, 
and professional fields. ^lumanities included all languages 
and literature, philosophy, religion, speech, fine arts, 
music, and dramatic arts. Social sciences included geog- 
raphy, anthropology, economics, history, government, 
psychology, and sociology. Natural sciences included all of 
the biological sciences, chemistry, phWg, geology, and 
niathematics. Professional fields includeTnttainly depart- 
ments of education, engineering, and business In addition, 
a total of about a dozen responses came from department 
heads in fields such as architecture, agriculture, forestry, 
home economics, veterinary medicine, and law. 

Table 1 includgs the number of departments in each 
group and at each university'level. There were 102 depart- 



Table 1. Number of Departments In the Study within Each 
Department Group and. at Each University Level 







Department Groups 






University Levels 


Professional 
Fields 


Social 
Sciences 


Humanities 


Natural 
Sciences 


Total 


Research Universities 


43., 


36 


36 


43 


158^' 


DoctorabGranting 
Universities . 


26 


27 


27 


42 


122 


Compretiensive 
Universities and Colleges 


47 


39 , 


41 


46 


173 


Total 


116 


102 


104 


131 


453 



mentb in the social sciences, 104 in the humanities, 131 in 
the natural sciences, and 1 16 in the professional fields For 
each of the three levels 'of universities determined by the 
Carnegie classification scheme, there were between 26 and 
47 departments in each of the four discipline groups, thus 
allowing department group by university level interactions 
to be investigated. " 

Each criterion in the questionnaire was responded to on a 
fi\e-point seale as ft)llows:iH>t available, not a factor, riiinor 
factor, major-faLtor, and extremely critical factor The aver- 
age response for each criterion was computed for each de- 



parlmenl group an J univurbit) levol. Analybib of vanance 
and canonical dibciiminanl lunation analybib were Ubcd in 
the study to investigate response differences. 



gated, these interactions were significant only in the evalu- 
ation of scholarship or research performance, canonical dis- 
criminant function analysis was used to illustrate this in- 
"Teractionr ™ 



Results . 

The results are presented in three sections: the criteria used 
for evaluating overall fatult) performance, the suurcos of 
information for evaluating toathmg, and the kinds of in- 
formation msed- for judging scholarship or research perfor- 
manc^^jn.each section the data were 'inalv/.ed^) level of 
university and b) department subgroup. Although djlfer- 
ences for department groups within uath of the universit) 
J levels (department b) level interaLtiun) were also investi- 



Criteria in Assessing TotalFaculty Performance 

The cjuestionnain? included 13 general criteria that might be 
used in evaluating faculty members. These are listed in Fig- 
ures 1 and 2 and in Tables 2, 3, and 4, along with the sum- 
maries o_f_thejcespoiises_of^department-^chairmen. Class- 
room teaching, cjualit) of publications, and personal qual- 
ifications (academic degrees and professional experience) 
were most frequently reportedas major or extremely critical 



Figure 1. Average weight giyen to various criteria for evaluating total faculty performance, by university level 



Not a 
Factor 



0 



Criteria 

Classroom teaching' ' 
Number of pubficaiions'" 
Quality of publications' ' 



Research and'or creative activity 
(independent of publication) 



Supervision of student research, including 
Serving on master's and doctoral committees"* 



Student advising'^ 

Campus committee worK, service to colfege' ' 



Activity tn professional societies 
(hold office* edit journaL^ptc ) 



Public or community service* * 
Competirig lob offers"* 
Consultation (government, business, etc )" 
Personality factors I 

• \\ 

Personal qualificatior^ (academic degrees 
professional experience, etc )" 

•p 05 
'-•p Of 



Minor 
Factor 




Extremely 

Critical 

Factor 



< r 



Key, 

Research Universities 
Ooctofal'Graniing Universities 
Comprehensive Universities and Colleges 



factors in judging faculty members. As indicated injure 
\, however, departmenl cliairmen nt'tlie three lovels^nSin- 
_siiluUoj:\s..gave-di£feitiat->mphase^.QU5ioxl3-^^ 
criteria. Less weight v.as placed on 'classroonv teaching or 
personal qualifications, in the Research Universities than in 
Doctoral-Granting Universities or Comprehensive Univer- 
sities and Colleges. Qn the other hand, the reverse was true 
for quality of publications, which, not surprisingly, was 



judged more important in the Research Universities than at 
either of the other two university levels. Chairmen in Com- 
prehensive^^ 

emphasis on quality of publications, but also gave it essen- 
tially the same uejght as the number of publications and as 
college ser\ice (for example, campus committee work). 
Other criteria in which universities differed according to 
Carnegie classification were the following: supervision of 



Figure 2. Average weight given to various Criteria for evaluating total faculty perforinance, by departmental subgroup 



Not a 
Factor 



Criteria 

'Classroom teaching 
Number of publications* * 
"Quality ot pubffcatiops"* 



P?search andor creative activity 
(independent of publication) 



Minor 
Factor 



Extremely 
Critical 
Factor 



Supervision of student research, including 
serving on master s and doctoral committees 



Student advising** 

Campus committee work, service to college' 



Activity in professional societies 
^ (hold office, edit journal, etc)* 



Public or community service* " 
Competing |ob offers" " 
Consultation (government, business, etc)** •* 
Personality factors 

Personal qualifications (academic degrees, 
^professional experience, etc.)" 




*p ' 05 
••p - .01 



Key: 

Professional Fields 



. — Social Sciences 

Humanities 
- • — Natural Sciences 
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Table 2. Current and Preferred Criteria for Evaluating Total Faculty Performance 



Percentage indicating Extremely CriticaiTactof or Not a Factor 



Current Use an(l ^Importance in 
Personnel Recommendations 





Professional 
Fields " 


Social ^ 
Sciences 


' Humanities - 


Natural - 
Sciences 


Professional 
Fields 


\ Social 
Sciences- 


Humanities 


Natural 

Science's 




N- 




N= 


102 


N- 


104 


N= 


-13j 


N= 


116 


N= 


=102 


N= 




N= 


131 






Not 
a 

Fact. 




Not 
a 

Fact 














Not' 
a 

Factr 




Not 
a 

Fact. 




Ext. 
Crit 


Ext. 
Grit. 


Ext.^ 
Crit. 


a 

Fact. 


Ext, 
Crit, 


a 

Fact. 


Ext. 
Crit, 


a 

Fact. 


Ext. 
Crit. 


a 

Fact. 


l^t. 
Crit. 


Ext. 
Crit. 


Classroom teaching 


38 




35 


1 


--44- 


1- 


31 


0 


59 


0 


46 


1 


69 


0 


44 


0 


nUllluCI Ul pUUIIvaliUlld 


1 A 
lU 


0 


1 c 
19 


o 


1J 


b 


12 


"2 


4 


] 


8 


1 


b 


7 




1 


Quality of publications 


OA 

20 


9 


41 


4 


oo 


3 


36 




34 


0 


55 


^ 0 


CO 

52 


1 


49 


1 


Researph and^or creative activity 

\iMUcpCIIUCMl Ul puuiiLaliU'ij * 


12 


3 


8 


7 


17 


6 


18 


2 






15 


3 


23 


0 


21 


1 


oupciVidiuii Ul diuuviii icdCdiuiii 

including serving on master's and 
doctoral committees 


6 


8 


5 


9 


8 


8 


6 


5 


16 


2 


8 


6 


15 


2 


15 


2 


Student ad Vising 


8 


11 


2 


24 


6 


M\ 


3 


17 


13 


1 


10 


8 


11 • 


7 


6 


5 


Campus committer work, service 
to college . 


2 


5 


3 


9 


5 


4 


2 


9 . 


5 


3 


1 


6 


^4 


3 


2 


5 


Activity in professional societies 
(hold office, edit lournal. etc,) 


2 


8 


2 


10 


4 


.5 


2 


8 


3 


3 


5 


5 


6 


3 


4 


4 


Public or community service 


2 


24 


2 


"29 


2 


36 


1 


47 


3 


16 


3 


25 


2 


27 


0 


36 * 


Competing job offers 


1 


53 


7 


31 


4 


47 


4 


44 


1 


58 - 


3 


38 


2 


43 


2 


49 


Consultation (government, 
business, etc:) 


1 


23 


1 


45- 


1 


47 


0 


50 ' 


2^ 


21 


2 


35 


0 


43 


1 


40 


Personality factors " 


' 5 


15 




10 


21 


4 


18 


9 


16 


1 


33 


7 


25 


2 


21 


Personal qualifications 
(academic degrees, professional 
experience, etc.) 


27 


3 


32 


7 


28 


6 


23 


8 


23 


3 


30 


6 


32 


5 


21 


8 



Irnportance Each Shoul6 Have in 
Personnel Recommendations 



s^tudent re5>tMrLh (grLMtcs>t cmph*is>is> in RubeofLh Univer- 
sities), student tuKisin^ lgrLM test emphasis in Comprehen- 
sive Universities tind Colleges), und Lompetin^ job offers 
(most important m Researeh Universities). 

Figure 2 illustrates the differences in emphasis on each 
enterion aecordrng to departmental subgroup, Publieations 
(qualitv and number) were more important in the natural 
and soLial seienees than m the profesbional schools and the 
humanities. Student advising, pubhc service, and consulta- 
tion were given more weight in the professional depart- 
ments than m the other three department groups. 

Current and preferred criteria for evaluating faculty 
members for decisionson prumotio;is are given in Fables 2 
and 3 (b) departments) and in Table 4 (b> university level). 
The absolute .differences between current and preferred 
emphases vv ere *also investigated and have been taken into 
account in thf' following discussion. The percentages of de 



partmenl he«ids reporting each criterion as an "extremely 
critical factor" or "not a factor" are reported in Table 2. As 
this table indicates, there are not only variations between 
the department subgroups but, to some extent, within the 
subgroups as well, For example, 41 percent of the social 
science department heads reported quality of^publications 
to be an extremely critical factor and 4 percent reported it 
mj/ to be a factor. In g»*neral. Table 2 indicates that quajjty of 
publications was a little more frcH:juently chosen as a critical 
factor than teaching was in the social sciences and natural 
sciences, while the reverse was true in the professional 
fields and the humanities. Criteria that chairmen preferred 
to emphasize more were quality of publications, supervi- 
sion of student research, advising students, and, to some 
lextent, unpublished research or creative activity (Tables 2 
t.nd 3). Hie number of publications should, according to the 
chairmen, be a less critical factor. Several criteria were not 



'Table 3. Ranking of Current and Preferred Criteria for 
Evaluating Faculty for Promotbn, Salary Increase, or Tenure, 
by Departmental'SQbgroup^ 





Current Use 


Importance Eaph 




and Importance 


Sftot/y^Have 




P 


s 


H 


N 


P 


s 


H 


N 


OldSSivOiii luduiiiiiy 


1 


1 


1 
1 


1 ^ 


1 
1 


0 

c 


1 


2 


t\tiimhpr fif-oubiir^tions 


5 


4 


c 
0 


0 


7 


c 
0 


7 
1 


u 


Quality of publications 


3 


2 


2 


L5 


2 


1 


2 


1 ' 


Research and/or creative 


















activity (independent of 
















- 


publication) 


4 


6 


4 


6 


3 


"4 


4 


4 


Supervision of student research. 


















inciuoinQ sciviny uii liidbici o 


















and doctoral committees 


a 

K) 


J 


7 


5 


5 


6 


5 


3 


Student advising 


7^ 


9-5 


9 


10. 


6 


7 


6 


7 


v>ampus cuiiiniiucc "Ui IS, ocivjuc 


















to college 9 


Q 

y 


7 
/ 


D 


Q 
0 


q 




3 


g 


Activity in professional 


















societies (ho}d office, edit 


















journal . etc ) 


8 


8 


8 


7 


8 


9 


9 


8 


Public or community service 


11 


12 


11 


12 


11 


10.5 


11 


11 5 


Competing |bbi offers 


13 


11 


l2 


11 


13 


12 


12 


13 


Consultation (government. 






13 


13 


12 


13 


13 




business, etc.) 


12 


ia 


11 5 


Personality factors 


10 


9.5 


10 


9 


10 


10-5 


10 


10 


Personal qualifications 


















(academic degtees, professional 


















experience, etc.) 


2 


3 


3 


4 


4 


3 


2 


5 



«P - Professional Fields. N - 116 (rtio - 97) 
S Social Sciences. N 102(rho-.94) 
H ^ 'Humanities; N - 1^4 (rtio - 94) 
N ^ Natural Sciences; l31'(rho - 90) 

Rank difference correlations between current importance and preferred im- 
portance are shown m parentheses ^ 
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Table 4. Ranking of Current and Preferred Criteria for 
Evaluating Faculty for Promotion, Salary Increase, or Tenure, ^' 
by University LeveP 





Q{irrQr\i Use 


Importance Each 




and Importance 


SftouW Have * 




AIL 


1 


II 


III 


Ail 


1 


II 


III 


Classroom teaching 


1 


2. 


1 


1 


1 


2 


1 


1 


Number of publications 


4 


3 


4 


5 


6 


6 


6- 


7 


Qualityj)lpublications _ ^ 


2 

— 


1 


2 


3' 


2 


J 

— 


2 

— . 


2 


Research and/or creative 


















activity (independent of 


















publication) 


5 


5 


5 


4^ 


4 


3 


4 


4 


Supervision of studerit research. 


















including serving on master's 


















and doctoral committees 


6 


6 


6 


8 


5 


4 


5 


6 


Student advising 




9 


10 


7 


7 


7 


7 


^ 5 


Campus committee work, service 


















to college « 


7' 


9 


8 


6 


9 


9 


9 


8 


Activity in professional 


















societies (hold office, edit 


















journal, etc.) 


8 


7 


7 


9,5 


8 


8 


8 


9 


Public or community service 


11 


12 


11 


11 


11 


12 


12 


11 


Competing |0b offers 


12 


9 


13 


13 


13 


11 


13 


13 


Consultation (government. 








12 


12 


13 


11: 


-ir 


business, etc.) 


13 


13 


12 


Personality factcirs 


10 


M 


9 


9.5 


10 


10 


10 


10 


Personal qualifications 


















(academic degrees, professional 
experience, etc.) 


3 


4 


3 


2 


3 


5 


3 


3 



'Departments grouped by university classification as follows: 

I - Research Universities; N - 158 (rho ^ .92) 

II - boctoral-Grantmg, Universities: N^ 122 (rho - .95) 

III Comprehensive Universities atid Collegesj N ^ 173 (rho .95) 
Rank dilference correlations between current importance and preferred 
impdrtance are shovvn in.parentheses. 



factors in cvaUintinj; laculty jjcVlornianco in many 
departments— in particular, conipetinjj job otters,, consulta- 
tion, public or community service, and .personality lactors. 
Moreover, none of these deserved more weight, accorduig 
to the respondents. I ho respondents also were generally 
satished witl\ the great importance given to personahiuni- 
ifications.. * 

Table 4 prescMits a ranking ot the current and preferred 
criteria bv level of universitv. Respondents \\\ Research 
Universiti^^ thought that quaht\ of publications followed 
bv classroom teaching should be the top two criteria, as 
indeed has been the case \\\ current pr^^clice. I iowever, the\ 
would put less emphasis on the number (^1 publications and 
more on research or creative activit\ independent ot publi- 
cation and on student advising, iiitluding the supervision 
K\\ student research. Respondents \\\ the second and third ^ 
level universities, where ^Hassroom teaching .was ranked 
lirst as a current and preterred aiterum. would aKo like to 
^»ee student advlsmg and supervision ot student reseanh 
emphnsi/.'»d. 

8 * ^ . 



Evaluating Teaching Performance 

A sdmmarv of the responses to 15 possible sources of in- 
formal ion\used to evaluate the teaching performance of in- 
dividual faculty members appears in Figures 3 and 4 and in 
Tables 5, 6, and 7, , 

* Chairinen evaluations,.*systematic i^tudent ratings, and 
colleagues' opinions were generally the most influential in- 
dicators of teaching performance. Least used were vid- 
eotapes of classroom teaching, the long-term follow-up of 
students, alumni ratings, colleague ratings based on class- 
room visits, and student examination performance. The 
weight given to several of the indicators differed by level of 
universitv (Figure d). Systumattc student ratings, forevam- 
ple, were less important at Compiehensive Universities 
and Colleges than ot Research or Doctoral-Granting Uni- 
versities. Deans' evaluations and self-evaluations were 
least impi»rtanl at* R^siMrch-Universities Teaching im- o 
provement activities, such as participation m in-sorvice 
programs or faculty development practices, tended to carry 



Figure 3: Average weight given to various sources of information for evaluating teaching performance, by university level 



Sources of 
Information 



Systematic student ratings** 

Informal student opinions ^ 

Colleague ratings based 
. ortctassrcom visits 

Colleague.opinlons* 
Student exam performance 



Content of course syllabi 
and exarf» nations 



Chairman evaluation 

Oean evaluation** 

Committee evaluation 
Self-evaluation or report* * 

long-term fOt'iOW'Up of 
students' performance 

Alumni opinions or ratings 

Popularity of elective 
courses (e g., enrollment)' 

Videotape of classroom teaching 



Teaching improvement activities 
(pamci'patiori in workshops, 
in-service programs, etc.)** 



•p < .05. 
*'p..< S>\ 



Not a 
Factor 




Major 
Faclorc> 



Extremely 
Critical 
Factor 



Key: 

Research Untyersitles 
Doctoral-Granting Universities 
Compreljensive Unlversities and Colleges 



more vveiglil in prornutionb .il ihc ConiprehunbUL Unuur- 
bities tind Cullcgub tlicin at thu oilier two IcvcLs. 

rhu depiirlniLMU subgroups differed on onl) ill rue of ihe 
tetiehing indiLiilor*? (Figure 4), Aeeording lo ihe LhiUrnien, 
the protesMonii! deparlinenb einphcusi/ed deiin.s e\iilua- 
lioiib tiiuMiiduhing uiipro\enienl dtliMlieb more ihiin did 
the olljer three dcptirtnient groups. FetiLhing uiipro\ement 
tietivilies were gi\e(i the least v\ eight in so<.icii seienee de- 
partments. Both social science and humanities departments 
put slighlK more emphasis on enroHinent in eleLti\e 
Lourses as an indieator of teaLhing pel lorniiinLe than did 
the professional or natural seienee departments. 



Within en eh of the department subgroups there were also 
Lonsiderable differences in the emphasis gi\ en to several of 
the mdieators (Table 5), For example, 21 percent of the so- 
Ltal SLienee departments reporte^i sNstematie student rat- 
ings as an extremelv critiuil factor, v\hereas FLpercent said 
the) were not ii factor. Committee evaluation varied even 
more, u ith, almost ecjUiilK high numbers of departments 
ineluding It as iin ext^remely eritical factor or not a factor at 
all. 

Fables \ 6, and 7 report the preferentes of department 
heads in e\aluciling teaehing. Changes in the absolute val- 
ues of current \s. preferred weights were also inspected. 



Figure 4. Average welghtgiven to various sources of Information for evaluating teaching performance, by depa 



Sources of 
Information 

Systematic student ratings 



Informal student opinions 

- o 

Colleague ratings based 
on classroom visits 



Colleague opinions 

Student exam performance 

Content ot course syllab 
and examinations 

Chairman ©valuation 
Dean evaluation* ' 
Committee ©valuation 
Self'evatuatton or report 



Lor^'term foHow-up of 
students' performance 



Alumm opinions or ratings 

•Poptjiarjty of elective 
courses (e g , enrollment)' " 

Videotape of classroom teaching 



Teaching improvement acuviiics 
(participattph in workshops, 
in-servico programs, etc.)** 



•*p- 01 



Not a 
Factor 



Major 
Factor 




Critical 
Factor 



Key: 

— Professional Fields 
• «— ~ So(^al Sciences 

Humanities 
• «M • Nataral Sciences 



Allhough the respondtMils said lh«U sy.slemiUic bliKlLMU rat- 
ings should have slightly more importance than the\ cur- 
-rentlv do, few would make them a critical factor ( Fablt^j* 5 
and 7). Colleagues' opinions should be less important, but 
colleague ratings ot teaching based on classroom visits, cur- 
rently not a lactor in many departments, should be given 
much grenter emphasis. I he content of course syllabi and 
of examinations should also be more important, moving 
from seventh- or eighth-ranked m currenf emphasis to 
titth-rankel/ in preterred importance. Ihe popularit\ of 
elective courses, respondents generally agreed^ should re- 
ceive relatively less importance, although they preferred to 



gi\ e more emphasis to long-term follow-up of student per- 
formance (Tables f)*and 7). Several sources of information, 
the respondents s\iid, should have more importance in 
evaluating teaching, \ilthough there was little change in 
their relative position as indicated by the rankings. These 
were, student examination performance, ilumni opinion*? 
or ratings, teaching improvement activities, and videotapes 
ol clasSTOom teaching. Apparently -department heads 
would like to attach more importance to a wider variety of 
information than they now' tlo. Whether it is feasible to, do 
so with some of the criteria— for example, alumni ratings of 
teachers—is another question. 
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Table 5. Current and Preferred Sources of Information for Evaluating Teaching Performance 



Systematic student ratings 

Jnlormal student opinions 

Colleague ratings based on 
' classroom visits 

Colleague oijinions 

Student examination performance 

Content of course syiiabi and 
examination 

Cliairman evaluation 

Dean evaluation 

Committee evaluation 

Self'evaluation or report 

Long-term follow-up of 
«<students' performance 

Alumni opinions o<^ ratings 

Popularity of elective courses 
(e.g., enrollment) • 
Videotape ol classroom teaching 

"teaching improvement aulivities 
(participation m workshops, 
in-service programs, etc ) 



Percentage. Indicating Extremely Critical Factor and Not a Factor 



Current Use and Importance 



Professional 
Fields 

N-116 



Ext 
Crtt 



Not 

a 

Fact 



15 
1 

2 

12 
4 

2 
23 
22 
22 

3/ 

2 
1 

1 

0 V 



5 

'6 

50 
9 
31 

38 
5 
'2 
20 
37 

59 
48 

43 
29 



Social 
Sciences 

N-102 



. Not 
Ext a 
Crit Fact 



21 



4 

10 
4 

1 

16 
10 
24 
♦1 



11 

7 

54 
8 
"38 

38 
10 
28 
25 
45 

61 
57 

25 
64 

49' 



Humanities 
N-104 



Ext 

cm 



Not 
a 

Fact 



24 



9 
17 

4_ 

5 
24 
13 
27 

4 

6 
0 



7 
8 

47 
6 
22 

31 
8 

27 
18 
40 

51 

.57 

23 
71 

48 



Natural 
Sciences 

N^13l 



Ext. 
Crit 



21 



5 
8 
2 

2 
17 

7 
24 

3 

2 
0 



Not 

a 

Fact 



9 
5 

34 
3 
32 

32 
5 
24 
16 
35 

47 
45 

40 
01 

45 



Importance Each Should Have 



Professional 
Fields 
N^116 



Ext, 
Crit. 



16 
1 

9 
9 
Z 

6 

25 
15 
18 . 
6 

6 
3 

0 
4 



Not 
a 

Fact 



1 
9 

19 
4 
38 

8. 

2 
11 
12 
19 

23 
34 

34 
47 

14 



Social 
Scieri^es 

N-102 



Sxt. 
Crit 



Not 
a 

F^ct 



20 
1 

10 
5 



2 
13 

3 
21 

2 

5 
0 

\ 
2 



8 
14 

39 
11 
38 

15 
10 
30 
19 

'38 

38 
38 

25 
50 

31 



Humanities 
N»=in4 



Ext 
Crit. 



Not 

a.^ 
.Fact. 



23 
1 

11 
13 
5 

11 

24 
•9 
24 
6 

10 
4 

0 
1 



3 
10 

22 
7 
31 

13 
7 
25 
13 
21 

20 
32 

27 
51 

22 



Natural 
Sciences 

N;^131 



Ext. 
Crit. 



15 
3 

8 
7 
2 

5 
14 
4 

22 
5 

7 
2 

0 
2 



Not 
a 

Fact. 



' 4 
14 

19 
7 

.32 

14 
5 

26 
11 
27 

20 
31 

38 
56 

27 



Table 6. Ranking of Current and Preferred 
Sources of Information for Evaluating Teaching 
Perrormance, by Departmental Subgroups 
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Table 7. Ranking of Current and Preferred 
Sources of Inforrpation for Evaluating Teaching 
Performance, by University LeveP^ 





Current Use 
and Importance 


lmpotia«ice Each 
Snoutd Have 




AW 


I 


11 


^ Ml 


All 


1 


II 


m 


Sy$i»>matic stud$m rairnqs ' 


25 


2 


2 


4 


1 


1 


1 


2 


j'\fo*mai stutfem oD»n(ons 




5 


5 


6 




$ 


7 


75 


Co!fwgy€ latmgs based of» 
ciassfootn visits 


12 


12 


10 


12 


7 


7^ 


6 


9 






1 


3 


2 


3 


3' 


3 


4 


C^effOfmance 


t1 


0 


12 


11 . 


12 


9 


13 


1? 


Cof^ie'it ot course syifabJ 
and exammaiK^ns 


7 


7 


8 


7 


5 


5 


S 


5 


Chairman evaigaiion 




3 


1 


1 


2 


2 


2 


1 


Oe^j> evsfuanon 




8 


6 


& 


8 


n 


9 


6 


Commutes evaUiai.on 






4 


3 


4 


4 


4 


3 


Sei!'€v3*uation teuton 


9 


to 


9 


85 


11 


14 


10 


10 


students peMormance 


14 


14 


14 


13 


9 


8 


8 


11 ' 


Alumni oiiir.jons ratings 




13 


13 


14 


14 


12 5 


12 


14 


PopuUnty oJ €feCl«ve ^ 
cours^^s (e g enroHmentl 


8 


6 


7 


10 


•U 


10 


14 


13 


Videotape oi cJassroom 
teacftiog 




IS 




15 


15 


15 


15 


15 


leachidg improvemeni activiites 
ipjrlfCJpjtion m vvcrJ<SfiOPS 
ii'servtce piograms etc » 


10 


il 


n 


86 


10 


12 6 


U 


75 



Oepari'nenis grouped oy unjversity ctassifica^<on as foHows 
I Research umvetSitiei H 158 79) / 
H Di^t'trtral^Grantiog t/ni^^'rstiies N 1 *? *tf>ct 7*i 
ii( Cvmprehens've Uf^'VHr<,i((e«i and Co^t?ge"i> N I73 rrhj 93> 

^Rjnk diffe'ence corretattons betAeen current imporiaice ar»d preferred tn* 
tiQiWb are sftowr^ in parer^lheses 



Evaluating Scholarship and Research 

Figures 5, 6, 7, and Sand Tables 8, 9, and 10 bummari/e the 
responses of department heads to the kinds of information 
used in evaUialing scholarship or rohearLh performaiKc. In 
general, the number ot articles published in qualitv jour- 
nals, the number of books oi which the laculty member is 
the sole or senior author, and.the qnalitv of one s research 
and publications as judged by peers at the institution are 
the three most important types of mtormation. used. But 
there are; important variations between departmental 
^groups and university levels. The Research Universities 
lend to put morJemphasis on the judgment of peers— both 
from within an^l outside the institu'tion— than do either 
Doctoral-Granling Universities or Comprehensive Univer- 
sities and Colleges (Figure 5). Comprehensive UniversUjes 



and Colleges put the leaJit emphasis on most of the criteria 
listed, with the Doctoral-Granting Universities falling be- 
tween them and the Research Universities. 

The departmental groupJi also differed in how they 
evaluated research and scholaiship (Figure 6) The profes- 
i,ional departments, compared with those in the arts and- 
s^.ieiKes, placed less emphasis on peer judgments and on 
the number of quality publications. Essentially, however, 
the w«i\ in which departments evaluated research in- 
teracted with the level of the university— that is, the de- 
partments differed according to the level of the university 
1 his interaction can be.illustrated canonical dis- 

criminant function analysis of ^ (the four de- 

partments within each of the :vei:> of universities) 

and of-the 16 kinds of information jsed to assess research 
performance. The analysis provides an optimal discdmjna- 



Figure 5. Average weight given to.various kinds of Information used to evaluate 
scholarship or research performance, by university, level „ 



Kinds of 
4nforniation 



Not a 
Factor 



Minor 
Factor 



Major 
Factor 



Extrernely 

Critical 

Factor 



Number of: 1 
Publications in all professional journals 

. Articles in qua//f/ journals' ' 

Unpublished papers or reports * ^ 

Papers at professional meetings 

Citations to published materials'* 

Boo_ks as sole or^senior author" " 

c ^ 

Books as junior author.or editor* 

Monographs or chapters in books'* 

Quality of scholarly research and 
publications as judged by: 
Peers at the institution* ' 

*» 

Peers at other- rnstiiutions ' * , 
Departmerit chairman* • 



it 



Doan 



Self-evaluations 
Grants or fOnding received 
Referee or editor of professional purnal" 
^;Honors or awards from profession" " 
* 'p < :05 




Key: 

Rescjarch Universities 



/mm^ mm^_ mmm mmm mmm mm Doctoral-Granting Untversitios 

Comprehensive UxJjversities and Colleges 



liun.between Ihe 12 gnniixs on Ihu bahl^ diu lb kmdb of 
information, Ui?ing the niLMii ruhpon^i^b of jlcparlmunl 
heads as Uie unil of analy sLs. 

The jcsullb of Ihe canonical discriminanl tunchon 
annKsib, A'hich are bur.'manzud in Appendix B and m Fig- 
, iires 7 and . mdicalu U^; ^* diniLMiMonb of \aiialion 
among Ihe 12 groupb oii mdb ot infurmaUon. Ta^l}^ 

of these two. dimension dior about a third o( the j 

variance among the II ^ .p-> lui-a totai of <>7 per<^ent {A(>r 
pendix B). Ihe gri,, p tentruids are plotted on the d^yrim- 
ihant a\/.'S in Figure example, in the fii^Cire, 1 P 

*The tiiscnniinant fuft.tion weij^hls-^iven m ApjK'nthv H are difficult to 
intefpret. fherelore, in order lu better ilndcr^iaIul the rouUv the 
bet\voen'|if**«P> *.urielalioiia betv^^tn tht uri^tnal responst^ aiu! ihc dis 
criminant a\es were studied. Ihcse ctirreladons h.ive tlu* advantage of 
being treaied nuich like factor loadings of oSt?r% ed variables on orttiogonal 
- tactor a<eb— the vanontud dt>crim«\ant iunvtiuns (Lliti and KruN, 1^75), 



identifieb the pobition ol profesMonal departments in Re- 
^earLh Uni\erbitieb, 11 S reprebent^ social hcien^^e depart- 
-nient.s in IJoctoral-Granting Universities, and so forth. In 
addition to mese group «.cntroidb. Figure 7 includes 16 lines 
of \ectors originating from the center of the plot. These 16 
lines represent the \vj\ the 16 kinds of informat on used to 
e\aluate research and s^.holarship project into the discrim- 
nant space. The importanl features of these vectors aie 
their directions and tl\eir relati\e lengths. B\ \isualizing the 
direct perpendiLuIar projection of the 12 group centroids 
onto these \eLtors, one can get a feeling lor the relative 
ordering among the djj^xirtmehtuiniversity level groups as 
to how they evaluate research performance. 

A simplihed illustration ol the data appears in Figure 8. 
The two axes were rotated by visually estimating where 
the\ bestjit the canonical variates. The descriptions at the 



Flgure.6. Average weight given to various kinds of information used to evaluate 
scholarship or research performance, by departmental subgroup 



Kinds of ^ , Not a 

Information - * Factor 

Numt>er of: 1^ 
Publications in all professionat journals 

Articles in <7ua% journals** ' 

Unpublished papers'or reports 

Papers at professional rrieetings' 

Citations to, published materials 

Books as sole or senior author" 

' -Books as junior author or editor" 

^ Monographs ^r chapters in books" 

^ 'M 

Qualtty/of scholarly research and 
publications as>judged by: 
Peers at the instituiroh" 

Peers at other institutions ' ' 
Department chairman 

N 

' Dean;' 

.Self-evaluations 
Grants or funding received* * 
Referee or editor of professional }ournal 
Honors or awards from profession 
*p • - .05 



Minor 
Factor 




Extremely 
Critical " c 
Factor 



Key; 

■I Professional 'Fields 

• wm^ mm* mmm ,m SOCtat ScfenCCS 

Humanities 

« « MM • «^ • >M Natural Sciences 



.14 



/ 
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Figure?. Plot oruniversily level and departmental subgroup centrolds and the 16 kinds of Information used to evaluate 
■Scholarship and research In the space defined by the two largest canonical discriminant axes 



• 11! P 



Axis I 



Self-evaluation 



Grants 
received 



• 11 N 



• IIIH 



• HiS 



Papers at j 
professionai 
meetings 




• IN 



Citations 



Professional awards 



Articles in 
quality journals 

Int^rrialand 
external peer 
evaluations 



• 11 H 



Key: 

I = Research Universities ' 
I L= Doctoral-Granting Universities 
Ifl - Comprehensive Universities and Colleges 

P = Professional Fields 
S = Social Sciences • / 
' H = Humanities 
N = Natu?ahSciences 



Figure 8. Rotated canonical discriminant axes, kinds of information used to evaluate scholarship and research performance 



• ill P 



Dimension 2 

Evaluations by 
dean and self 



IIP 



• Ml 



• IMS 



Number of books and 
monographs published, 
papers at professional 
meetings 



• II H 



Dimension .1 
Grants received 




• UN 



• I N 



Peer evaluations - 

Articles in . 
quality journals 

Professional awards 



• IS 



• I H 



Key: 

I = Research Universities 

II = Doctoral-Grant ng Universities 

III = Comprehensive Universities and Colleges 

P= Professional Fields 

S = Social Sciences 

H = Humanities .^^ 

N = Natural Sciences 



ERLC 



IC 



15 



Table 8/ Information Used to Evaluate Scholarship or Research Performance 



Percentage-indicating Extremely Critical Factor and Not a Factor 





Current Use and Importance 


Importance Each Should Have 




\ 


Professional 
Fields 

N=116 


3ocial 
^ Sciences 

N==102 


Humanities 

N«:^104 


Natural 
Sciences ^ 

N=131 


Professional 
Fields 

N=116 


Social 
Sciences 

N=102 


Humanities 
- N=104 


Natural 
Sciences 

N=131 


Ext. 
Cnt 


Nnt 
A 

Fact. 


Ext. 
Crit. 


Not 
a 

Fact. 


Ext 
Crit. 


Not 

a 

Fact 


Ext. 
Crit. 


Not 
a 

Fact. 


-Ext. 
Crit. 


Not 

a 

Fact. 


Ext. 
Or*:, 


Not 
a 

Fact. 


Ext. 
Crit. 


Not 
a 

Fact. 


Ext. 
Crit. 


"Not 
a 

Fact. 


Number of: 


































Publications in all prcfessipnal 
journals 


21 


A 


14 


2 . 


14 


6 


18 


1 


11 


3 


14 


? 


13 


3 


15 


1 


Articles in quality journals 


29 


3 


46 


2 ' 


37 


3 


40 


1 


44 


1 


57 


1 


45 


1 


' 45 


1 


Unpublished papers or reports 


0 


23 


1 


24 


1 


18 


0 


22 


1 


17 


2 


16 


1 


12 


1 


23 


Papers at professional meetings 


3 


6 


4 


3 


13 


3 


3 


2 


3 


2 


4 


5 


13 


1' 


5 


2 


Citations to published materials 


5 


36 


2 


32 


1 


33 


4 


28 


' 7 


15 


5 


19 


' 4 


J 2 


8 


11 


Books as so\t or senior author 


25 


6 


53 


1 


54" 


4 


■ 15 


5 


26 


2 


53 - 


1 


59 


2 


. 15 


1 


Books as junior author or editor 


5 


8 


14 


1 


10 


6 


5 


-6 


' 5 


3 


14 


1 


. 8 


2 


6 


3- 


Monographs or chapters in* 
books 


4 


6 


18 


0 


17 


, 4 


6 


4 


7 


2 


20 


. 1 


^ 20 


2 


7 


2 


Quality of scholarly research and 


































publications as judged- by: 






























37 




PccfS=3t th.8 instilution 


15 


10 


36 


7 


36 


6 


36- 


3 


E 20 


3 : 


, 35 


6 


-_ 37 


1 


2 


Peers 'at other Institutions 


16 


22 


25 


14 


31 


15 


36 


10 


19 


5 


32, 


5 


38 


2 ' 


38 


2 


Department chairman 


14 


7 


16 


14 


19 


12 


10 


6 


12 


7 


17 


12 


15 


7 


12 


8 

26 


Dean 


13' 


15 


8 


32 


5 


29 


2 


27 


; 8 


15 


5 _ 


36- 


3 


26^^ 


1 


S( jvaluation^ • 


4 


43. 


1 


46 


2- 


51 


2 


42 




^ 30 




51 


4 


40 


. 2 


38 


Grants or funding received 


20 


8 


. 7 


11 


6 


21 


16 


1 


13 


4 




J1 


4 


11 


. 10 


3 


Referee or editor of professional 
journal 


0' 


12 


7 


8 


7 


8 


5 


7 


4 


4 


9 


2 


10 


.4- 


3 


. 4 


Honors or awards from profession 


14 

f 


6 


16 


2 


12 ' 


3 


18 


2 


21 


0 


20 - 


* 1 


. 14 


2 


-18 


0 



end of each axis or dimension summan/e whith of tlie 16 
variables secnvlo best detine the pole. The first dimension 
contrasts grants received willv llie numberof bookb pub- 
lished and papers presented at professional meelinjJis. v|Vs . 
indicated by their position turlhest out from the ce^^er, 
natural science departments in the first two levels of "uni- 
versities (I^esearch and Doctoral-Granting Universities), 
followed at some distance by third-levei natural science de- 
partments (Comprehensive Universities and Colleges) and 
first-level prolessional departments, were at the grants re- 
. ceived" end of tlie continuum. As Figures 7 and 8 further 
illustrate, ihe.humanitics departments in the top two levels 
of institutions were at the books-papers'* pole. The second 
dimension contrasts evaluations b> deans and b> the indi- 
vidual faculty member (seif-evalualion)iwilh peer and pro- 
fessional evaluations. The latter include judgments of re- 
search or publications by peers from »nsideand outside the 
institution, articles pubhshed in quality journals, awards 
fronv the profession, and, to a lesser extent, the number of 
citations to the individual's pubhshed materials and journal 
editorship. Evaluations by deans and self-evaluations 
were emphasized most m professional departments at sec- 
ond- and third-level institutions as well as in humanities ^ 
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departments in third-level universities. Peer and profes- 
sional evaluations (f6r examplb, articles in quality journals) 
vvere=emphasi7.ed--mosl in social science departments in 
first-leyej universities. To some extent, humanities and 
natural science departments in first-level universities also 
emphasized peer and professional evaluations. In sum, the 
assessment of research and scholarship, as Figures 7 and 8 
help to make clear, varies considerably depending on the 
type of department and level of institution. 

The current and preferred urfporlance of each criterion 
for assessing research and scholarship is summarized in^ 
Tables 8, 9, and 10. Tibbie 8 also includeshhe percentages of 
departments that use:|Ke criterion as an extremely critical 
factor or not at all. The quality of publications iJs judged by 
peers, department heads, or deans was 'extremely impor- 
tant in man) departments, yet wms not used as.a criterion in 
many others. For example, 25 percent of the re*^pondenls in 
the social science departments reported peers at other in- 
stitutions to be critical in judging scholarship, while 14 per- 
cent did not call on these peers as judges at all, 

Department heads in the Research Universities were 
generally satisfied with the relative and absolute weights 
the\ use iir evaluating research or scholarship. Those in the 
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Table 9. Ranking of Current and Preferred Criteria for 
Evaluating Research or Scholarship Performance, 
' by Depart mental Subgroupi 





Cuffent Use 
and Importance 


Importance Each 
Should Have 




•P 


S 


H 


N . 


P 


S 


H 


N 


Number of: 


















Publications in all professional 
journals 


'I 

o 




4 


5 


8,5 


8 


8 


6 


• Articles in gya//f/ journals 


0 

c 


0 


c 


1 
1 




] 


2 


] 




1 A 




1 } 
10 


Id R 


IS 


Id 


14 


IS 


Donate 3t nmfpccmnsl mpptinn^ 


in 


1 1 

1 1 


Q.J 


1 1 
1 1 


] ] 


12 


7 


]] 


Pitatinnc in mihlichprt mslPrisl^i 
vlldllUllo lU puUlljIICU lllalCiiaid 


1 A 
lO 


1 <^ 
ID 


10 






1*1 

1 0 


IS S 


1 o 


Rnnbc cntp nr Qpninr Pttthnr 


1 

1 


1 
1 


1 

1 


7 


2 


2 


] 


5 


D^OKa as_ junior duuioi uf uuiiur 


0 


a 

D 


in 


10 


10 


7 


] ] 


10 


|VlClilv\Jldpild Ui vMu^lCid llf'^UUU'VO 


9 


4 




8 

# 


6 


5 


5 


8 


Qualip/ of sch.olarly research and 
publications a^ judged by: 


4. 
















Peers at the Institution 


5.5 


3 


,3 


2 


3 


3 


3 


2 


Peers at other institutions 


12 


8 


8 


3 


5 


4 


4 


3 


Department chairman 


7 


9 


9 


9 


8.5 


9 


9 


9 


Oean 


11 


14 


14 


M.5 


13 


15 


15,5 


14 


Self-evaluations 


16 


16 


16 


16 


16 


16 


10 


16 


Grants or .funding received 


4 


10 


12 


_ 4 


7 


10 


13 


7 


Referee or editor of professional 
lournai 


13 


12 


11 


12 


12 


11 


12 


12 


Honors or awards from profession 


55 


7 


6.5 


6 


4 


6 


6 


4 



*P ^ Professional Fields. N- 1 16 (rho 82) 
S - Social Sciences. N ^ 102 (rho ^ 94) 
H ^ HumanUies; N 104 (rho = 89) 
N - Natural Sdtences. N 131 (rho 97) 

Rank difference correlations bet^veen . current importance and preferred im- 
portance are shown in parentheses 



Table 10. Ranking of Current and Preferred Criteria for 
Evaluating Research or Scholarship Performance, 
by University LeveP 





Current Use 
and'Importance 


Importance Each 
StwuldM^n 




Ad 

All 


1 


n 
II 


111 
Ul 


All| 1 


li 
ii 


Ill 
III 


Numoer of 


















Publications in all professional 
journals 


4 


5 


5 S 


2 


7 


7 


11 


7 


Articles m quality journals 


1 


1 


1 


3 


1 


1 


1 


1 


Unpublished papers or reports 


14 


15 


14 


14. 


i5 


15 


15 


13 


Papers at professional meetings 


11 


M 


12 


7 5 


11 


95 


12 


9 


Citations to published materials 


15 


13 


15 


155 




13 


14 


14- 


Books as sole 0{ senior author 


2 


3 


2 i 


1 


2 


4 


2 


2 


Books as junior author t)r editor 


10 


9 


10 


9 


9 


9-5, 


9 


8 


Monographs or chapters m books 


6 


7 


4 


7 5 


g 


5 


6 


6 


Quality of scholarly research and 
pubrications as judged by 


















Peers at the institution 


3 


2 


3 


.4 . 


3 


2 


3 


'3 


- Peers at other institutions 


7 


4 


8 


12 


•4 


3 


4 


5 


Department chairman 


8.5 


8 


7 


;o 


8 


8 


/ 


12 


Oean 


a 


14 


13 


13 


14 


14 


16 


15 


Self-evaluations 
Grants orfiinding received* 


16 


16 


fe ' 




1& 


16 - 


46. 


J6. 


8,5 


10 


9 


6 


!0 


Ti 


8 


10 


Referee or editor of professional 
journal 


12 


12 


11 


11 


12 


12 


10 


11 


Honprs or award§ from profession 


5 


B 


5r5 


5 


5 


5^ 




4 



Oepartments grouped by university classificalion as follows 

I "f^esearch Universities, N 158 (rho - 98) 

II 'boctoral-Granimg Universities; N 122 (rho 92) 

III ConjQrehensive Universities ^nd Colleges. N 173 (rho 83) 

RanK dillerence correlalio.io between curren'i importance and ^refened im 
portpnce are shown m parentheses 



Comprehensive IniverMtius oiul CoHuge^ thuu^ht that 
more weight should be given to jiulgments m.ulc b\ peer^* 
at other institutions and in the niimber ol pitbhcalionh in 
qiiniily journals. Most respondents, regardless of ie\e(of 
university or department, prcterred to put relatneU Je^js 
emphasis i>n the sheer number i>t publitatk>os in all l\,pes 
pt journals;: I he natural stieme and proiussional depart- 
ments also preferred to put shghth less u eight on grants 
received than they currently do. 

Discussion 

Teaching, research, and service are usualh enumerated as 
the three functions ol most universities and, therefore, pre- 
sumably the three major concerns ol tacull\ members. But 
the responses given by 453 department heads from 134 uni- 
versities indicate that public or \ini\eD»ily service is gener- 
ally given little importance m evAilualing lavult> members 
tor decisions reg«ifdmg tenure, salar\, and promotion. The 



Research Uni\ ersilies, with large Ph:D programs and 
hea\ \ tiivancial support for research, emphasi/e research, 
as m^ghl be expected. Teaching, though, according to the 
respondents, is a close second in importance At the 
Doctoral-Griiiting Univcrsitie.s and the Comprehensive 
Uni\ersiliesand CoJleges,. leaching was ranked first in im- 
pixrlance, followed .closely by research. Personal qualifica- 
tions, such as having an advanced degree imd the appro- 
priate experience, wre the only other criteria considered 
major factors in advancement decisions. Therefore, in addi- 
tion to community and publii: service, student advi,singand 
service to the ijislilulion were also secn=as relatively minor 
factors. In short, this study suggests that once the appro- 
priate credentials are in hand, most universities and de- 
partments evaluate most faculty members as researchers- 
sJioiars a'lul classroom teachers, with \arying emphasis 
depending on the level of the university orlhe type of the 
department. Of course, it is important to note that the 
chairmen were responding to Hie criteria as they applied to 
their entire department, several pointed out that individual 
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faculty members m the department frequently ha\e \ar>ing 
responsibihliesand that the importance of an) crit<.\riun de- 
pends on what those responsibilitiub are. Some Lhiiirmen 
also added that the significance of the vanoub criteria de- 
pends on the level of the advancement in quebtion and that 
each candidate was expected to be outbtnnding in at lea>t 
one area (usually research or teaching). 

Whether leaching does, in fact, receive as muLh weight in 
promotion and tenure decisions as chairmen report or 
whether they are merely paying lip service to itb importance 
is difficult to know. There is reason to doubt. The facult) in 
the previously reported Oregon State study (Thorne, Stott, 
and Beaird, 1976) viewed publicationb ab the mobt influen- 
tial piece of evidence in decisions^pn advancement. It rnay 
be that research activity would have ranked ahead of tlabb- 
room teaching for all levels of universities (not jUbt the Re- 
Search Universities) if the questionnaire had listed a single 
criterion (as with classroom teaching) inbtead of three bepa- 
r<7te kinds of information (number of publications, quality 
of publications, and research andor creative acti\it\ inde- 
pendent ot publication). This may have diffused the em- 
phasis that research and bcholarbhip activity actuall) re- 
ceives. 

The results of this study also suggest an increase in the 
u£ie of some of the more systematic and tangible kindb of 
.-/jevidence for evaluating teaching performance. In compari- 
son with the Astin and Lee survey a decade earlier, sys- 
tematic student ratings and, to some extent, the content.of 
*course syllabi and examinations are used moreJrequently. 
Other recent studies, as discussed earlier, havQ noted a 
jssimihir increase in the use of student ratings for tenure and 
promotion decisions. Surprisingly, according to the re- 
sponses of department heads in this study> .student ratings 
were even moreimportant in the Research Uni\ersUiesand 
the Doctoral-Granting Universities than in the Comprehen- 
sive Universities and CQlIeges. Possibly the larger bi/e of 
the Research and Doctoral-Granting Universities fostered 
^ the use of formal student ratings as evidence of teaching 
effectiveness. ^ ' 

Although there*appears to be an increase in the use of at 
-^leasbsome substantive evide nce for evaluating teaching, 
^ apparently its use falls far short of what department heads 
think IS needed. They would like to see even n^pre em- 
phasis on systematic student ratings and an evaluation of 
the content of course syllabi and examinations. They also 
believe that formal colleague ratings based on classroom 
visits and;a long-term follow-up of how students perform 
should-get-more-weight than they currently do. But both 
formal colleague ratings and the assessment of long-term 
student outcomes are difficult measures to obtain in any 
systematic and usable wqy. Colleague ratings based primar- 



*, 

il) on Jassroom observation would in most instances not 
be reliable enough to use in making decisions on tenure and 
promotion— at least not without faculty members investing 
considerable time in visitations or training sessions (Centra, 
1975). And the assessment of student learning years after 
the students have con\pleted a course, while possibly an 
ideal indicator of the long-term effects of a ^course or 
teacher, is unwieldy to administer; moreover, the effects 
are difficult to attribute to a specific teacher. An assessment 
of eiid-of-course learning, although more manageablp, is 
also subject to misinterpretation when used in tenure and " 
promotion decisions (Centra, 1977). No doubt, however, 
student learning is an, important criterion in assessing 
teaching, and it may be that colleagues (or a department 
committee) could help review and judge pre-post a\3hieve- 
nient gains, studehi projects, or other evidence of coui::se 
-outcomes. 

In comparison with the evaluation of teaching, the evalu- 
ation of re.search makes possible the use of much more 
tangible evidence,. although judgments still play an impor- 
tant role in the evaluation. Articles, books, awrirds from the 
profession, and, in the natural sciences, grants received are 
the most important kinds Of information used. Qu«ility of 
research, according to the chairmen,, is and should be the 
primary consideration. Therefore, in most academic fields 
the number jDf articles in quality journals is of prime impor- 
tance, along-with the judgments of research or scholarship 
made by peers «7t the institution. In fact, chairmen feel that 
jnore use"5hould be made of peers at other institutions as- 
additional reviewers of the? quality of a faculty member's 
research or scholarship.^ 

But in general, the evaluation of research and scholarship 
depends very much on the type of department and the level 
of institution,^The number of books and paper's- produced, 
for example, is especially important in humanities depart- 
ments at the top t\<'o levels of universities, but not at the 
third-level. Peer judgments of research and the number of 
articles in quality journals are important in social science 
departments in the Rlesearch Universities-but not in social 
science departments in the Comprehensive Universities^ 
and Colleges. These variations point out that the criteria^ 
used to evaluate"research-and scholarship-should be set at 
the department level rather than at the institutiohal 
level— or, for that matter, by the discipline as a whole. In- 
deed, the same might be said for evaluating total faculty 
per/ormahcd: the individual departments probably need 
some flexibility in setting their own criteria and standards 
of performance. Department procedures should be consid- 
ered within the general framework provided by university 
polijcies and,.as department heads in this study indicated^ 
should be based on as much hard.evidehce as possible. 
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Graduate, .Record- Examinations. Board 



PRINCETON. NEW 



JERSEY OeS^O t.AREA CODE 609 921-9000 



IN AFFILIATION WITH 

The Assocjation of Groduato Schools 

The Council of Graduate Schools 



1975-1976 



Sanford S. Elberg 
University of California at Berkeley 
Chairman 



May 1976 



Richard Armttage 
Ohio State University 

May Brodbeck 
University of ibvva 

Beverly Cassara 
federal City College 

David R7Deener 
fulane University 

I. Wesley glhoit 
Fisk University 

Wytze Gorter 
University of Hawaii at Manoa 

Arthur F Jackson 
North Carolina Agricultural 
and Jechmcat State University 

Lylo V.Jones 
University of North Carolina 
atChapel Hill 

Sterling M. McMurj^n 
University of Utah 

J. Suyd Page 
Counctt of Graduate Schools 
(ox officio) 

.GeorQ4 J. Resnikoff 
California State University 
at Hayward^ 

Lo'^ene Rogers 
.University of Texas 
et Austfn 

Ben Rothblatt 
^ University of Chicago 

Harry H Siilec 
University of Florida 

Oonsid J. Whue 
Boiton College 

W. Dexter Whitehead 
University of Virginia 
atCharlottesvi^fe 



Maryonn A. Lear 
«r«tiiry to the Board 



Dear Colleague: * * ' 

We are undertaking, a survey of faculty evaluation procedures and 
nee'd' your assistance* The purpose of the survey is to investigate 
what departments are doing and what they think ought to be <ione/When*^ 
evaluating the performance of faculty members* 

We hope you will be able to complete the enclosed questionnaire and 
return it to your graduate dean as soon as possible. He or she will 
forward your responses to researchers at Educational Testing. Service 
who will summarize the results. Questionnaires are being distributed 
to a sample of department chairmen at about 150 universities. 

The questionnaire asks about the methods, being used in your depart- 
ment to evaluate faculty for promotion, salary increase, ^or tenure. 
Please indicate the importance given to each criterion when evaluating 
performance, including both teaching and scholarship or research 
performance. We would also like to know how you think various sources 
of information, ought to be used- We re.cognize that final decisions 
are" frequently made beyond the department level; however, recommenda- 
tions by departments are critical- Therefore, this questionnaire 
focuses on procedures that are usejd to arrive at departmental 
recommendations- " • ^ 

i 

Because the response format may not allow you to adequately describe 
what Is done at your department, in faculty evaluation (or^what you 
think should be done), we have also provided space for your add!-- ^ 
tional commeats- The survey report will not identify individuals or 
institutions by name- . < 

Thank you for your help with the survey- A report of the results 
will be sent to you at your request and we trust it will be of 
interest to^ ypu- 



Sincerely, 




Sahford S- Elberg / / 
Chairman 



Maryann A. Lear 
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SURVEY OF DEPARTMENTAL PRACTICES IN EVALUATING 
FACULTY PERFORMANCE 

I- Listed below are criteria that might be taken into account in evaluating faculty 
members for promotion, salary increase, or tenure. We would like. you to give 
your best judgment about: . ^ 

A. how important each criterion is in personnel judgments currently being made 
about faculty members within the department; (Column A) 

B. how important you think each criterion should be . That is, given; the goals 

of your department and institution, how much weight ought to be placed on each 
. factor? (Column B) Respond as follows: 

1 =''Not a factor 

2 = Minor factor 

3 = Major factor * 

4 = Extremely critical factor 

0 = Not available 



Current use and 
importance -in personnel 
recommendations 

(Circle one response in 
each row) 



B 

Importance each should 
have in personnel 
recommendations 

(Circle one response 
in each row) 



Classroom teaching 1 


2 


3 


4 


0 " 


1 2 


3 


4 


f Number of publications ■ 1 


2 


3 


4 


- 0 


1 2 


3 


4' 


' Quality of publications 1 


2 


3 


4'^ 


0 


1 2 


3/ 


■4 


* /*• 

Research and/or creative activity 
independent of publication ) -T 


2 


3 


4-. 


0 


1 2 


3 


4 ■ 


Supervision of student research, including 
'-. serving on. masters and doctoraT coimiittees " 1 


2 


3 


4 


0 


I 2 


3 


4 


• Student .advising ' , 1 


2 


3 


4, 


0. 


1 2 


' 3 


4 


Campus- committee work, service to coTlege 1 


. 2 


3 


4' 


0 


1 2 


3 


4 


Activity in professionaT societies (hold 
'7~^ off ice, edit journal , etc, \ 1 


2 


3 


4 


0 


1 2 


3 


4- 


Public or community service „y 1 


2 


3 


4 


0 


T 2 


3 


4 


Competing job offers • 1 


2 


3 


4 


.0 


1 2 


3 


4 


Consultation (government, business, etc..) 1 


2 


3 


4 


0 


1 2 


3 


4 


^ Personality factors 1 


2 


3 


4 


0 


1. 2 


^3 


4 


Personal qualifications (academic degrees, 
process ionaT experien^ce, etc.) 1< 


^ 2 


'3 


4 


0 


1 .2 


3 


4 


Other (specify) 
















■ <f- > _ ' ■) 


2 


3' 


4 


0 


1 2 


' 3 


4 



Comments about above criteria: 
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Listed below are sources of information that might be taken into account when evalu 
atinq the teaching performance of individual 'f acuity members in connection with 
promotion, salary increases, or tenure recommendations. .WKat is the gq^.^";PJ^- 
tanceVf each in your department and what importance do you think each should .have? 

1 = Not a factor ' ^ 

2 = Minor factor 

3 = Major factor 

4-=. Extremely critical factor ^ - 
0 = Not available 



Indicators of 
teaching performance 



Systematic student ratings 

Informal student opinions 

Colleague ratings based on 
classroom visits 

Colleagues' opinions 

Student examination performance ^ 

Content' of course sjllabi and 
examinations 

" Chairman evaluation 

Dean evaluattoti 

Committee-'^evaluation ' ' 

Self-evaluation or report 

Long term follow up 'of how ^• 
strfdents perform 

Alumni opinions or ratings 

Popularity of elective courses ^ 
(e.g. , enrollment) 

Video-tape of classroom teaching 

Teaching improvement activities 
(participation in workshops, in- 
service programs, etc.) 

.Others (specify) 



Current use and 
importance in 
evaluating 
teaching performance 

(Circle one response 
in each row) 

2 3. 4 0 

2, 3 4 6 



i2 
2 
2 

2 
2 
2 
2 
2 

2 
2 

2 
2 



3 
3 

'3 

3 
3 
3 
3 
3 

3 
3 

3 
3 



4 
4 
4 

4 
4 
4 
4 
4 

4 
4 

■ 4 

■ 4 



0 
0 
0 

0" 
0 
-0 
0 
0 

0- 

0 

0 
0 



■ Importance each 

■ should have in 
- evaluating 

teaching performance 

(Circle one response 
in each row) 

•1 2 3 4- 



1 



2- 

2 

2 

2. 
2 
2 
2 
2 

2 
2 

2 
'2 



3 
3 
3 

'3 
3 
3 
'3 
3 

3 
3 

3 
3 



4 
4 
" 4 

4 
4 

^ 4 
4 

,.4 

4 
4 

4 



Comments about above indicators: 



III. Listed below are kinds of infoririation that might be used to evaluate the scholar- 
ship or research performance of individual facul^fe^ members. Which ones are now 
used in your department when, making personnel decisions? Which ones do you think 
should be used? ' ' , . - . 

1= Npt,a factor 

2 = Minor factor ' . 

3 = Major factor 

4 = Extremely critical factor 



Kinds of information 



Number of: 
Publications in all professional 



0 = Not available 

Current use and 
importance in 
*' evaluating scholarship 
or resear^ performance 

'(Circle one response 
in each row) 



Importance each 
should have in 
evaluating scholarship 
or research performance 

(Circle one response 
in eatch row) 



journals 


1 2 


3 


4 


0 


1 


2 


3, 


■4 


Articles in quality journals 


1 2- 


3 


4 ■ 


0 


1 


2 


3 


4 


Unpublished papers or reports 


1 2 


3 


4 


0 


1 


2 


3 


4 


Papers at professional meetings 


1 2 


3, 


4 


0 


-1 


•2 


3 


4 


Citations to published, material s 


1 2 


3 


4 


0. 


-1 


2 


3- 


4 


Books as sole or senior author 


1 2 


.3 


4- 


0 


1 


2 


3 


4 


.Book5 as junior author or editor 


1 2 


3 


4 


0 


1 


2 


3 


.A 


Monographs or chapters in books 


1 2 


3 


4 


0 


1 


2 


3 


4 



Quality^of scholarly research and 
publica'tions as judged by: 

Peers ,at the institution 

Peers at other institutions 

• Department chairman 

Dean- 

Self-evaluations 

Grants or funding received 

Referee or editor of professional- 
journal 

Honors or awards from profession ^ 
Others- (specify) 



2 
2 

V 

2 
2 
2 



3 
3 
3 
3 
3 



2 3 
2 3 



4 
4 

'4, 
4 
4 



4 
4 



0 
0 
0 

;o 

0 



0 
0 



2, 

2- 

2 

.2 

2 



'2 
2 

1 -2 



3 
3 
3 
3 
3 



3 
3 



4 

4 

4 

4' 

4 



4 
4 



Comments about above information: 
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Please use the space below to comment orv^ practices in your department or at 
your institution that may not be adequately reflected in your responses thus 
far/ Are there-practices that you think should be followect instead? 



Name 



Title or Rank_ 

Department 

^Address 



University_ 



Highest degree o'ffeVed by the department:^ __ 

Number of ful-1 -time faculty members in 'the department: ____ — ^ . 

Check here if you would like a qppy. of the final results. _ 

Someone in your graduate ^^ean's, off ice is coordinating the distribution and 
collection of these questionnaires. Please return your completed form to this 
person. • ' • 



Thank you for your assistance- 



John Centra 
Mary Jp Clark 
Project Directors 
Educational: Testing Service 
Princeton, New Jersey 08540: 
(609) 921-900D 



24 



Appendix B 



CanonicarDiscriminant Funclios* Analysis of Information 
Used to Evaluate Scholarship and Research' 



Discrimiuattl Fumiion 
I 

2 

3' 
4 
5 
6 
7 
8 
9 
10 
Ml 



RiK>l 

.5130 
.4492 
.1548 
,1050 
.0729 
.0591 
.0283 
.0280 
.0233 
.0084 
.0039 



VVilk's Lambda Criterion = .287 for F (176, 3909) = 3.25 

Test for residual after removing: 

First discriminant function 

First two discriminant functions 

First three discriminant functions 
V First four discriminant functions 

First ;(ive discriminant- functions 

First six discriminant functions 

First seven discriminant functions 

First eight discriminant functions 
^ First nine discriminant functions 

First ten discriminant functions 



V 



t'Cmmdatwc % Trace 
35.5 
66;5 
77.2 
84.5 
8V.6 • 
. 93.6 - 
95,6 
97.5 
99.1 . 
99.7 
100.0 , 



'Chi-square 
365.95 
203.09 ^ 
139.91 . 
96.07 
65.16 
. 39.94 
27.68 
15.55 
5.42 
1.72 



.150.^ 
126 . 
104 

84 , 

66 

50 

36 

24. 

14 
6 



Discriminant Function Weights Scaled for Unit 
Within-Groups Variance on Original Responses 



Discriminant Function 



Variable 



Number of: 

Journal publications 
Articles in quality journals 
Unpublished pa persf reports 
Papers at professional meetings 
Citalions to materials 

* Books as sole or senior nulj^or 
Books as junior author or cdijor 
Monographs or chapters 

QuaUh/ of iivrk In/: 

Peers at the institution ' . 
Peers at other institutions 
D epa r t m en t c ha i r ma n 
Doan 

Self-evaluations 
Grants or funding received 
J(Hir|pLeditor or referee i 
^Pmfes.sional awardsHionors 



/ - 


// 


/// ' 


.179 


.028 


.025 


.015 


.413 


.233^ ^ 


-.099 


-.096 


-.157 . 


-.137 


_ 224 


" -.376 


.003^^ 




-.209 


-.568 


-.453 


' .265 


.234 


.222 


.256. 


-.111 


-.037 


.159 


-.201 


^' .287 


-.,ai5 


-.117 


^ .259 


-.314 


^.088 


-.180 


.20U 


.132 


-.09a^ 


-.113 


,156* 


-,152 


.087 


.451 


.211 . 


;334 


-.035 


-.05^5 




-.048 


.064 


^ MS 



, 'The .iiilhur is gMtcful to Allen Yalt.-. lor hi» help wilh ihe Mnonilal (Ii5cnmln.inl .in,ilyMS'> 
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